feat(map-efficient): #303 Slice 5b — real concurrent dispatch (flag-gated, default-off)#308
Conversation
…retries config (#303 Slice 5b.0)
…e in get_wave_step (#303 Slice 5b.0)
…d/verify-clean/reconcile) (#303 Slice 5b.1)
…h (phantom-parallelism detector) + parallelism.json (#303 Slice 5b.2)
…spatch harness (#303 Slice 5b.3)
…ors batch-split (#303 Slice 5b.4)
…ay rollback/restart (#303 Slice 5b.5)
…-discard note (dispatch_mode==concurrent) (#303 Slice 5b.6)
…nement, monkeypatch-fail, AST-import, no-telemetry, baseline) (#303 Slice 5b)
…GE) + activate dormancy-guard tests; make check green (#303 Slice 5b)
📝 WalkthroughWalkthroughAdds opt-in concurrent Actor dispatch for parallel waves, with config gating, wave-group lifecycle tracking, abort/retry coordination, parallelism classification, updated docs, and expanded tests. ChangesConcurrent Actor Dispatch — Slice 5b
Sequence Diagram(s)sequenceDiagram
participant Agent as MAP Agent
participant Orch as get_wave_step
participant Gate as compute_dispatch_gate
participant Runner as run_concurrent_wave
participant Merge as merge_wave_worktrees
participant Abort as abort_wave_group
Agent->>Orch: get_wave_step(branch)
Orch->>Gate: compute_dispatch_gate(branch)
Gate-->>Orch: dispatch_mode / reason / concurrency_enabled
Orch-->>Agent: wave step response
Agent->>Runner: run_concurrent_wave(group_ids)
Runner->>Merge: merge sub-batch
alt merge fails
Runner->>Abort: abort_wave_group(group_id)
Abort-->>Runner: rollback verified
Runner-->>Runner: retry or exhaust
else merge succeeds
Merge-->>Runner: merged ids
end
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja (1)
1-1: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRe-render the Codex copy
src/mapify_cli/templates/skills/map-efficient/efficient-reference.mdalready matches the Jinja source;src/mapify_cli/templates/codex/skills/map-efficient/efficient-reference.mdstill diverges and should be regenerated fromsrc/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja` at line 1, The Codex copy of the map-efficient supporting reference is out of sync with the Jinja source. Regenerate src/mapify_cli/templates/codex/skills/map-efficient/efficient-reference.md from src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja, using the same content as the matching template under src/mapify_cli/templates/skills/map-efficient/efficient-reference.md.Source: Coding guidelines
🧹 Nitpick comments (3)
tests/test_map_orchestrator.py (1)
5370-5436: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winAdd a mixed-wave regression for the active-wave gate.
VC3 covers “active wave is parallel” and “all waves are width-1”, but not
[["ST-001"], ["ST-002", "ST-003"]]withcurrent_wave_index=0. That gap lets a later parallel group incorrectly make the current sequential wave report concurrent dispatch.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_map_orchestrator.py` around lines 5370 - 5436, Add a regression test for the active-wave gate where the first wave is width-1 and a later wave is parallel, using compute_dispatch_gate and the existing WAVE_REASON_* constants. In test_vc3_flag_on_isolation_required_concurrent_and_sequential, add a case with execution_waves like [["ST-001"], ["ST-002", "ST-003"]] and current_wave_index=0, then assert the result stays sequential and does not return WAVE_REASON_CONCURRENT_GATED. Keep the focus on map_orchestrator.compute_dispatch_gate and the wave-reason symbols so the mixed-wave behavior is locked down.tests/test_concurrent_dispatch_harness.py (1)
54-59: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winAvoid leaking generated-script precedence into the whole test session.
sys.path.insert(0, ...)at module import time can change how later tests resolvemap_step_runner. Load the generated runner with a temporary path restore orimportlibunder a private module name.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_concurrent_dispatch_harness.py` around lines 54 - 59, The module-level sys.path.insert(0, ...) in test_concurrent_dispatch_harness.py is leaking generated-script precedence into the entire test session. Update the import setup around _SCRIPTS_PATH and the map_step_runner import to use a temporary path change or importlib-based loading under a private module name, then restore sys.path immediately after loading so later tests do not resolve map_step_runner from the generated scripts path.tests/test_map_step_runner.py (1)
13609-13614: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winExercise retry tests with destructive abort semantics.
The no-op
abort_wave_groupstubs let retry assertions pass even though the real abort deletes worktrees and group sidecar state. Add one retry test that uses real abort behavior or a fake that removes retry inputs.Also applies to: 14035-14041
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_map_step_runner.py` around lines 13609 - 13614, The retry coverage in map_step_runner is still using a no-op abort stub, so the test does not reflect the destructive behavior of abort_wave_group. Update the retry test around _fake_abort (and the related retry case) to use the real abort path or a fake that actually removes the retry inputs/worktree state, then assert the retry behavior with that cleanup in place. Keep the existing merge failure setup via _fake_merge_fail and the retry assertions, but ensure the abort semantics are exercised realistically.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/mapify_cli/parallelism_observability.py`:
- Around line 205-208: The `phantom_parallel` branch in
`parallelism_observability.py` is too eager: `skill_reported_concurrent` with
`same_turn_task_count <= 1` should only map to
`DISPATCH_OUTCOME_PHANTOM_PARALLEL` when there is no conflicting in-flight
evidence. Update the Rule 4 check in the dispatch classification logic to also
require `max_in_flight < 2` (or otherwise defer to the Rule 6 `unknown` fallback
when in-flight evidence contradicts the self-report), keeping the behavior
aligned with the existing `skill_reported_concurrent`, `same_turn_task_count`,
and `max_in_flight` rules.
In `@src/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinja`:
- Around line 2636-2649: The active wave should determine dispatch mode, but the
current logic in get_wave_step() relies only on
select_execution_strategy(branch, project_dir), which can mark
concurrency_allowed true based on any later color group. Update this decision to
inspect the current wave’s own parallelizability before returning
dispatch_mode="concurrent" or concurrency_enabled=True, and keep sequential when
the active wave width is 1 even if another wave is parallel.
In `@src/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinja`:
- Around line 19103-19118: The classifier currently only receives the
group-level base_sha, so mixed worktree bases never trigger the isolation check
in classify_dispatch(). Update the base SHA collection in the step runner to
gather each subtask’s worktree base SHA from the per-subtask records instead of
only _group_data["base_sha"], and keep feeding that list into classify_dispatch
so len(set(base_shas)) can detect mixed bases correctly.
- Around line 17448-17458: The retry path in merge_wave_worktrees is using the
same batch IDs after abort_wave_group has already deleted the group worktrees
and removed the wave_groups entry, so the next attempt has no inputs to merge.
Update the failure handling in the sub_batches loop to avoid retrying with
destroyed inputs: either perform the retry before calling abort_wave_group,
re-dispatch/recreate the actor worktrees before the next merge attempt, or abort
once and propagate the failure. Use merge_wave_worktrees, abort_wave_group, and
the failed/batches_merged bookkeeping to keep the retry flow consistent.
- Around line 15956-15992: Serialize the lifecycle update flow in the worktree
state helper so `seq` is assigned under a single writer lock instead of after a
plain read in the current lifecycle append path. In the block that reads
`state`, mutates `wave_groups[group_key]["lifecycle"]`, computes `max_seq`, and
calls `_write_worktree_state`, add synchronization around the read-modify-write
sequence to prevent two callers from reusing the same sequence number. Keep the
fix localized to the lifecycle event update logic and preserve the existing
monotonic `seq` behavior in the same function.
- Around line 17429-17439: Gate run_concurrent_wave() with
_concurrent_dispatch_enabled() before any batching or merge work so the
default-off execution.concurrent_dispatch config is respected. Add an early
return or error in the run_concurrent_wave flow, near the existing
branch_name/ids_sorted setup, when _concurrent_dispatch_enabled(pd) is false,
and keep the behavior consistent for direct CLI/coordinator calls. Use the
existing symbols run_concurrent_wave, _concurrent_dispatch_enabled, and
_wt_error to locate and implement the check.
- Around line 16115-16124: The terminal-check in the lifecycle sweep is too
permissive because the all(...) expression in the subtask completion logic only
evaluates list-valued lifecycle entries and skips missing or non-list slots.
Update the terminal-event validation in the map_step_runner template so every
declared subtask must be present and have at least one terminal event before
marking the group as terminal, using the existing lifecycle.values() /
all_terminal logic as the place to tighten the check.
- Around line 17338-17358: The abort/cleanup flow in the worktree rollback path
is removing the group entry even when rollback or discard operations may have
failed, which can hide an incorrect HEAD state. In the cleanup logic around
_wt_rollback(base_sha), discard_subtask_worktree, and record_group_lifecycle,
first capture and check the success/failure of rollback and branch cleanup, and
only delete the wave_groups entry after a verified successful rollback to
base_sha. If rollback fails, preserve the group state (including base_sha) so
verify_group_clean can detect the inconsistency.
---
Outside diff comments:
In
`@src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja`:
- Line 1: The Codex copy of the map-efficient supporting reference is out of
sync with the Jinja source. Regenerate
src/mapify_cli/templates/codex/skills/map-efficient/efficient-reference.md from
src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja,
using the same content as the matching template under
src/mapify_cli/templates/skills/map-efficient/efficient-reference.md.
---
Nitpick comments:
In `@tests/test_concurrent_dispatch_harness.py`:
- Around line 54-59: The module-level sys.path.insert(0, ...) in
test_concurrent_dispatch_harness.py is leaking generated-script precedence into
the entire test session. Update the import setup around _SCRIPTS_PATH and the
map_step_runner import to use a temporary path change or importlib-based loading
under a private module name, then restore sys.path immediately after loading so
later tests do not resolve map_step_runner from the generated scripts path.
In `@tests/test_map_orchestrator.py`:
- Around line 5370-5436: Add a regression test for the active-wave gate where
the first wave is width-1 and a later wave is parallel, using
compute_dispatch_gate and the existing WAVE_REASON_* constants. In
test_vc3_flag_on_isolation_required_concurrent_and_sequential, add a case with
execution_waves like [["ST-001"], ["ST-002", "ST-003"]] and
current_wave_index=0, then assert the result stays sequential and does not
return WAVE_REASON_CONCURRENT_GATED. Keep the focus on
map_orchestrator.compute_dispatch_gate and the wave-reason symbols so the
mixed-wave behavior is locked down.
In `@tests/test_map_step_runner.py`:
- Around line 13609-13614: The retry coverage in map_step_runner is still using
a no-op abort stub, so the test does not reflect the destructive behavior of
abort_wave_group. Update the retry test around _fake_abort (and the related
retry case) to use the real abort path or a fake that actually removes the retry
inputs/worktree state, then assert the retry behavior with that cleanup in
place. Keep the existing merge failure setup via _fake_merge_fail and the retry
assertions, but ensure the abort semantics are exercised realistically.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b0eecf5b-0e6b-4ae6-bc66-58088f203ac8
📒 Files selected for processing (30)
.agents/skills/map-efficient/SKILL.md.agents/skills/map-efficient/efficient-reference.md.claude/skills/map-efficient/SKILL.md.claude/skills/map-efficient/efficient-reference.md.map/scripts/map_orchestrator.py.map/scripts/map_step_runner.pyCHANGELOG.mddocs/ARCHITECTURE.mddocs/USAGE.mdsrc/mapify_cli/config/project_config.pysrc/mapify_cli/parallelism_observability.pysrc/mapify_cli/templates/codex/skills/map-efficient/SKILL.mdsrc/mapify_cli/templates/codex/skills/map-efficient/efficient-reference.mdsrc/mapify_cli/templates/map/scripts/map_orchestrator.pysrc/mapify_cli/templates/map/scripts/map_step_runner.pysrc/mapify_cli/templates/skills/map-efficient/SKILL.mdsrc/mapify_cli/templates/skills/map-efficient/efficient-reference.mdsrc/mapify_cli/templates_src/codex/skills/map-efficient/SKILL.md.jinjasrc/mapify_cli/templates_src/codex/skills/map-efficient/efficient-reference.md.jinjasrc/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinjasrc/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinjasrc/mapify_cli/templates_src/skills/map-efficient/SKILL.md.jinjasrc/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinjatests/_fake_task_tool.pytests/test_concurrent_dispatch_harness.pytests/test_map_orchestrator.pytests/test_map_step_runner.pytests/test_parallelism_observability.pytests/test_project_config.pytests/test_slice5b_leak_guards.py
…dings + tests) F1 phantom_parallel requires max_in_flight<=1; F2 dispatch gate keys on the ACTIVE wave width (not any wave); F3 lifecycle sidecar serialized via dedicated file lock; F4 reconcile requires terminal events for every declared subtask; F5 abort_wave_group preserves group state on failed rollback (head-mismatch); F6 run_concurrent_wave fails closed on concurrent_dispatch=off; F7 abort-once + needs_redispatch (no internal merge-retry over discarded worktrees); F8 classifier fed per-subtask base SHAs so isolation_violation is reachable.
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinja (1)
2645-2663: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winLoad
step_state.jsonfromproject_dir, not CWD.
compute_dispatch_gate(..., project_dir=...)reads config fromproject_dir, but the active-wave check uses.map/<branch>/step_state.jsonrelative to the process CWD. A caller operating outside the repo root can gate dispatch using the wrong wave state.🐛 Proposed fix
- state_file = Path(f".map/{branch}/step_state.json") + state_file = Path(project_dir) / ".map" / branch / "step_state.json" state = StepState.load(state_file)Also apply the same
project_diranchoring inselect_execution_strategy()where it loadsStepState, otherwiseconcurrency_allowedcan still be computed from CWD state.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinja` around lines 2645 - 2663, The active-wave dispatch check is reading StepState from a path rooted at the current working directory instead of the provided project_dir, so compute_dispatch_gate can use the wrong step_state.json. Update the StepState.load path in compute_dispatch_gate to be anchored under project_dir, and apply the same project_dir-based pathing in select_execution_strategy where it also loads StepState so concurrency_allowed is computed from the same repo state.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinja`:
- Around line 17565-17577: The retry counter in the abort path is being lost
because abort_wave_group() removes the wave group entry before
_read_worktree_state() can update it. Update the abort/retry flow in
map_step_runner.py.jinja so the attempt count is persisted in a separate state
location or captured before calling abort_wave_group(), then use that saved
value when computing attempts_remaining. Keep the logic tied to
abort_wave_group(), _read_worktree_state(), and _write_worktree_state() so
repeated redispatches continue incrementing correctly across failures.
In `@tests/test_map_step_runner.py`:
- Around line 14483-14489: The test around
map_step_runner.record_group_lifecycle is swallowing the expected OSError
instead of asserting propagation. Replace the try/except in this test with
pytest.raises(OSError) so the call to record_group_lifecycle(gk, "ST-Q01",
"started", branch) explicitly verifies the write error is raised while still
covering the release/finally behavior.
- Around line 14169-14199: The test is reimplementing the dispatch SHA
collection logic instead of exercising the real CLI/helper path. Update this
case in test_map_step_runner.py to invoke the actual runner entrypoint or helper
that uses record_dispatch_actual and then assert on the emitted classification
outcome. Keep the assertion tied to classify_dispatch only as a verification
target, but drive it through the real CLI flow so regressions in per-subtask
worktree record reading are caught.
- Around line 14277-14305: The current test only checks the first failure path
and does not verify that retry state persists across redispatches. Update
test_vc3_attempts_remaining_decrements_with_config_max_retries in
test_map_step_runner.py to exercise two consecutive failing calls for the same
wave group by reusing the same run_concurrent_wave / abort_wave_group flow, then
assert that attempts_remaining decrements again (for example from 1 to 0). Use
the existing map_step_runner symbols run_concurrent_wave, merge_wave_worktrees,
and abort_wave_group to confirm the stateful retry counter survives across
abort/redispatch cycles.
---
Outside diff comments:
In `@src/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinja`:
- Around line 2645-2663: The active-wave dispatch check is reading StepState
from a path rooted at the current working directory instead of the provided
project_dir, so compute_dispatch_gate can use the wrong step_state.json. Update
the StepState.load path in compute_dispatch_gate to be anchored under
project_dir, and apply the same project_dir-based pathing in
select_execution_strategy where it also loads StepState so concurrency_allowed
is computed from the same repo state.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ba071137-286a-446a-b385-97fa1c6ad896
📒 Files selected for processing (10)
.map/scripts/map_orchestrator.py.map/scripts/map_step_runner.pysrc/mapify_cli/parallelism_observability.pysrc/mapify_cli/templates/map/scripts/map_orchestrator.pysrc/mapify_cli/templates/map/scripts/map_step_runner.pysrc/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinjasrc/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinjatests/test_map_orchestrator.pytests/test_map_step_runner.pytests/test_parallelism_observability.py
🚧 Files skipped from review as they are similar to previous changes (6)
- src/mapify_cli/parallelism_observability.py
- tests/test_map_orchestrator.py
- tests/test_parallelism_observability.py
- src/mapify_cli/templates/map/scripts/map_orchestrator.py
- .map/scripts/map_orchestrator.py
- .map/scripts/map_step_runner.py
| abort_wave_group(group_key, branch_name) | ||
|
|
||
| # Read attempt count from sidecar (written by begin_wave_group / prior calls). | ||
| _st2 = _read_worktree_state(branch_name) | ||
| _wg2 = _st2.get("wave_groups") or {} | ||
| _grp2 = _wg2.get(group_key) if isinstance(_wg2, dict) else None | ||
| _attempts_used = 1 | ||
| if isinstance(_grp2, dict): | ||
| _attempts_used = int(_grp2.get("abort_attempts", 0)) + 1 | ||
| _grp2["abort_attempts"] = _attempts_used | ||
| _write_worktree_state(branch_name, _st2) | ||
|
|
||
| attempts_remaining = max(0, max_retries - _attempts_used) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Persist retry attempts outside the group entry before aborting.
abort_wave_group() deletes wave_groups[group_key] on success, so the read at Lines 17567-17575 usually finds no group and resets _attempts_used to 1 every failure. Successive redispatches will not exhaust max_wave_retries.
🐛 Proposed fix
- abort_wave_group(group_key, branch_name)
-
- # Read attempt count from sidecar (written by begin_wave_group / prior calls).
+ # Persist attempts outside wave_groups because abort_wave_group removes
+ # the group entry on successful cleanup.
_st2 = _read_worktree_state(branch_name)
- _wg2 = _st2.get("wave_groups") or {}
- _grp2 = _wg2.get(group_key) if isinstance(_wg2, dict) else None
- _attempts_used = 1
- if isinstance(_grp2, dict):
- _attempts_used = int(_grp2.get("abort_attempts", 0)) + 1
- _grp2["abort_attempts"] = _attempts_used
- _write_worktree_state(branch_name, _st2)
+ _attempts = _st2.setdefault("wave_group_abort_attempts", {})
+ if not isinstance(_attempts, dict):
+ _attempts = {}
+ _st2["wave_group_abort_attempts"] = _attempts
+ _attempts_used = int(_attempts.get(group_key, 0)) + 1
+ _attempts[group_key] = _attempts_used
+ _write_worktree_state(branch_name, _st2)
+
+ abort_wave_group(group_key, branch_name)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| abort_wave_group(group_key, branch_name) | |
| # Read attempt count from sidecar (written by begin_wave_group / prior calls). | |
| _st2 = _read_worktree_state(branch_name) | |
| _wg2 = _st2.get("wave_groups") or {} | |
| _grp2 = _wg2.get(group_key) if isinstance(_wg2, dict) else None | |
| _attempts_used = 1 | |
| if isinstance(_grp2, dict): | |
| _attempts_used = int(_grp2.get("abort_attempts", 0)) + 1 | |
| _grp2["abort_attempts"] = _attempts_used | |
| _write_worktree_state(branch_name, _st2) | |
| attempts_remaining = max(0, max_retries - _attempts_used) | |
| # Persist attempts outside wave_groups because abort_wave_group removes | |
| # the group entry on successful cleanup. | |
| _st2 = _read_worktree_state(branch_name) | |
| _attempts = _st2.setdefault("wave_group_abort_attempts", {}) | |
| if not isinstance(_attempts, dict): | |
| _attempts = {} | |
| _st2["wave_group_abort_attempts"] = _attempts | |
| _attempts_used = int(_attempts.get(group_key, 0)) + 1 | |
| _attempts[group_key] = _attempts_used | |
| _write_worktree_state(branch_name, _st2) | |
| abort_wave_group(group_key, branch_name) | |
| attempts_remaining = max(0, max_retries - _attempts_used) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinja` around
lines 17565 - 17577, The retry counter in the abort path is being lost because
abort_wave_group() removes the wave group entry before _read_worktree_state()
can update it. Update the abort/retry flow in map_step_runner.py.jinja so the
attempt count is persisted in a separate state location or captured before
calling abort_wave_group(), then use that saved value when computing
attempts_remaining. Keep the logic tied to abort_wave_group(),
_read_worktree_state(), and _write_worktree_state() so repeated redispatches
continue incrementing correctly across failures.
| # Replicate the runner CLI's per-subtask SHA collection logic (record_dispatch_actual). | ||
| group_sids = group_data.get("subtask_ids", []) | ||
| group_level_sha = group_data.get("base_sha") | ||
| base_shas: list[str] = [] | ||
| for sid in group_sids: | ||
| slug = map_step_runner._wt_slug(sid) | ||
| wt_rec = worktrees.get(slug) if (isinstance(worktrees, dict) and slug) else None | ||
| if isinstance(wt_rec, dict): | ||
| per_sha = wt_rec.get("base_sha") | ||
| if isinstance(per_sha, str) and per_sha: | ||
| base_shas.append(per_sha) | ||
| continue | ||
| if isinstance(group_level_sha, str) and group_level_sha: | ||
| base_shas.append(group_level_sha) | ||
|
|
||
| # Two members with different SHAs → classify_dispatch must return isolation_violation. | ||
| assert len(set(base_shas)) > 1, ( | ||
| f"Expected >1 distinct base_shas to trigger isolation_violation; " | ||
| f"got base_shas={base_shas!r}" | ||
| ) | ||
| from mapify_cli.parallelism_observability import ( | ||
| classify_dispatch as _classify, | ||
| DISPATCH_OUTCOME_ISOLATION_VIOLATION, | ||
| ) | ||
| outcome = _classify( | ||
| same_turn_task_count=2, | ||
| max_in_flight=2, | ||
| base_shas=base_shas, | ||
| skill_reported_concurrent=True, | ||
| ) | ||
| assert outcome == DISPATCH_OUTCOME_ISOLATION_VIOLATION, ( |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
This doesn't exercise the CLI path it claims to cover.
The test rebuilds base_shas inline and calls classify_dispatch() directly, so it will still pass if record_dispatch_actual regresses and stops reading per-subtask worktree records. Please invoke the actual runner CLI/helper and assert on its emitted classification instead of reimplementing the logic in the test.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_map_step_runner.py` around lines 14169 - 14199, The test is
reimplementing the dispatch SHA collection logic instead of exercising the real
CLI/helper path. Update this case in test_map_step_runner.py to invoke the
actual runner entrypoint or helper that uses record_dispatch_actual and then
assert on the emitted classification outcome. Keep the assertion tied to
classify_dispatch only as a verification target, but drive it through the real
CLI flow so regressions in per-subtask worktree record reading are caught.
| def test_vc3_attempts_remaining_decrements_with_config_max_retries( | ||
| self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch | ||
| ) -> None: | ||
| """VC3: attempts_remaining decrements; with max_retries=2 first call → remaining=1.""" | ||
| def _always_fail(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: | ||
| del ids, branch, kw | ||
| return {"status": "error", "ok": False, "kind": "WAVE_MERGE_CONFLICT", "message": "x"} | ||
|
|
||
| def _fake_abort(group_id: str, branch: Any = None) -> dict[str, Any]: | ||
| del group_id, branch | ||
| return {"clean": True, "aborted_group_id": "fake"} | ||
|
|
||
| monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _always_fail) | ||
| monkeypatch.setattr(map_step_runner, "abort_wave_group", _fake_abort) | ||
|
|
||
| repo = self._make_repo_with_dispatch_enabled(tmp_path, max_retries=2) | ||
| monkeypatch.chdir(repo) | ||
| monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") | ||
|
|
||
| result = map_step_runner.run_concurrent_wave( | ||
| ["ST-001", "ST-002"], "test-branch", repo | ||
| ) | ||
|
|
||
| assert result.get("kind") == "WAVE_ABORTED" | ||
| assert result.get("needs_redispatch") is True | ||
| # First call uses attempt 1 of 2; 1 attempt remaining. | ||
| assert result.get("attempts_remaining") == 1, ( | ||
| f"expected attempts_remaining=1 after first failure; got {result}" | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
This test never proves retry state survives across redispatches.
run_concurrent_wave() updates abort_attempts only after abort_wave_group(), and abort_wave_group() removes the group entry on a normal abort. That makes the cross-call decrement path fragile, but this test only asserts the first failure. Please drive two failing calls against the same group and assert the counter drops again (for example 1 -> 0) so the stateful retry contract is actually covered.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_map_step_runner.py` around lines 14277 - 14305, The current test
only checks the first failure path and does not verify that retry state persists
across redispatches. Update
test_vc3_attempts_remaining_decrements_with_config_max_retries in
test_map_step_runner.py to exercise two consecutive failing calls for the same
wave group by reusing the same run_concurrent_wave / abort_wave_group flow, then
assert that attempts_remaining decrements again (for example from 1 to 0). Use
the existing map_step_runner symbols run_concurrent_wave, merge_wave_worktrees,
and abort_wave_group to confirm the stateful retry counter survives across
abort/redispatch cycles.
| # The write will raise — record_group_lifecycle should propagate the error | ||
| # but release must still have been called. | ||
| try: | ||
| map_step_runner.record_group_lifecycle(gk, "ST-Q01", "started", branch) | ||
| except OSError: | ||
| pass # expected — the write is patched to raise | ||
|
|
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Assert the write error instead of swallowing it.
This currently verifies the finally path, but not the stated propagation contract. If record_group_lifecycle() starts catching the OSError, the test still passes. Wrap the call in pytest.raises(OSError) so the test checks both behaviors.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_map_step_runner.py` around lines 14483 - 14489, The test around
map_step_runner.record_group_lifecycle is swallowing the expected OSError
instead of asserting propagation. Replace the try/except in this test with
pytest.raises(OSError) so the call to record_group_lifecycle(gk, "ST-Q01",
"started", branch) explicitly verifies the write error is raised while still
covering the release/finally behavior.
Summary
#303 Slice 5b — activate REAL concurrent Actor dispatch in
/map-efficient: when a wave's color group is parallel-ready, the operator emits NTask(actor)blocks in one assistant message. Flag-gated and default-OFF (execution.concurrent_dispatch=false) → byte-identical to 5a. Slice 6 (default flips) is out of scope.Implemented per the llm-council deep verdict (conv
d5ff5181, 5b round, tracked on #303), which split 5b into 7 sub-slices (5b.0–5b.6). 10 subtasks via/map-efficient; every Monitorvalid/approve; independent final-verifier PASS; fullmake checkgreen (3207 passed).What changed
execution.concurrent_dispatch(bool, default false) +max_wave_retries(int 3, [1,10]) config + clampcompute_dispatch_gate— config-driven fail-closed gate inget_wave_step. Gate =concurrent_dispatch ∧ concurrency_allowed ∧ isolation≠off; flag-false short-circuits first (no probe — HC-1 byte-identity); hard-aborts (DispatchGateError) when flag on + isolation off (HC-3, never silent-degrade).WAVE_CONCURRENCY_ENABLEDkept dormant.begin_wave_group/record_group_lifecycle/verify_group_clean/reconcile_orphan_groups(idempotent, read/record-only)record_dispatch_actual+classify_dispatch— coordinator-owned phantom-parallelism detector. Clock-freemax_in_flightfrom sorted lifecycle-event replay; evidence hierarchy (worktree-SHA proves isolation, not concurrency); self-report never authoritative. Activateswrite_parallelism_report→parallelism.json(concurrent path only).threading.Barrier(N)fake Task deadlocks if the host is secretly sequential (no wall-clock); golden parsing + anti-phantom negatives.run_concurrent_wave— batch-split bymax_actors(now live, [1,8]); each sub-batch merged atomically via the existingmerge_wave_worktrees(HC-4, all-or-nothing).abort_wave_group— idempotent whole-group rollback reusing_wt_rollback(clean -fd -e .map …, neverclean -fdx);run_concurrent_wavebounded restart (max_wave_retries),WAVE_RETRY_EXHAUSTED→ escalate-to-human, no auto-restart.dispatch_mode==concurrent), sequential default text unchanged (HC-1).make checkgreen.Hard constraints (all proven by tests; final-verifier confirmed)
make check-renderclean; runner/orchestrator standalone (nomapify_cliimport)._wt_rollbackreused, no rawclean -fdx,.map/survives abort.make check(3207 passed) green.Key residual risk (by design)
Same-turn N-Task emission is LLM behavior the Python layer cannot force or directly observe — green CI proves the plan, not the act. The
phantom_parallel/same_turn_but_host_sequentialclassifier (ST-003) is the post-deployment detector and ships with 5b.Follow-up (not in this PR)
concurrent_dispatch/wave_mode/worktree.isolationdefaults after a soak. No shadow-mode rollout.Part of #303. Builds on 5a (#306).
Summary by CodeRabbit
execution.concurrent_dispatch.execution.max_wave_retries).